Universal Access To All Knowledge
Home Donate | Store | Blog | FAQ | Jobs | Volunteer Positions | Contact | Bios | Forums | Projects | Terms, Privacy, & Copyright
Search: Advanced Search
Anonymous User (login or join us)
Upload

Reply to this post | Go Back
View Post [edit]

Poster: Administrator, Curator, or StaffNemo_bis Date: Feb 2, 2014 4:17am
Forum: faqs Subject: Check for hash (md5 or sha1) to search item or verify S3 upload

I'm trying to make my uploads more robust and one thing I should really do is a file hash check. As far as I know, https://github.com/kngenie/ias3upload doesn't support it.
Two questions:

1) Does IA'a S3 support Content-MD5 / ETag headers to verify an upload? The ia-wrapper library now uses them https://github.com/jjjake/ia-wrapper/issues/48 but they're not mentioned in http://www.archive.org/help/abouts3.txt .

2) Can I search all items for a hash to check if a file was already uploaded? This would be useful in particular to avoid uploading duplicate files where I don't know the previous uploads' item identifier. I tried something like the licenseurl search https://archive.org/about/faqs.php#263 but it fails, I suppose that kind of metadata is not indexed yet? Or maybe I'm not using the correct syntax. Example: https://archive.org/details/utzoo-wiseman-usenet-archive -> https://archive.org/search.php?query=%2Fmetadata%2Fmd5%3A94b855519a152e3138f4fda5259d4443

Reply to this post
Reply [edit]

Poster: Administrator, Curator, or StaffJeff Kaplan Date: Feb 2, 2014 9:48am
Forum: faqs Subject: Re: Check for hash (md5 or sha1) to search item or verify S3 upload

the abouts3.txt info is pretty minimal and dated. there is more detailed info at https://github.com/vmbrasseur/IAS3API#internet-archive-s3-api-documentation

the current search engine does not support finding hashes but we hope to have that included in the near future.

Reply to this post
Reply [edit]

Poster: Administrator, Curator, or StaffNemo_bis Date: Feb 2, 2014 10:27am
Forum: faqs Subject: Re: Check for hash (md5 or sha1) to search item or verify S3 upload

Ah, fantastic! I'll replace all links to the old docs which I left around and study the new ones a bit.